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ABSTRACT 



Policies and practices in the areas of identification, 
assessment, and recognition of nonf ormal learning in the European Union (EU) 
were reviewed. The review focused on national and EU- level experiences 
regarding the following areas and issues: recognition of the contextual 
nature of learning; identification of methodological requirements for 
assessing and recognizing nonformal learning; and institutional and political 
requirements. Special attention was paid to the following experiences: (1) 

the German and Austrian dual system approach; (2) the Mediterranean approach 
of viewing methodologies for assessment and recognition of nonformal learning 
as tools for quality improvement; (3) the diverse approaches of the Nordic 
countries; (4) the National Vocational Qualifications approach in the United 
Kingdom, Ireland, and the Netherlands; (5) the ’’opening up" of diplomas and 
certificates in France and Belgium; and (6) EU-level initiatives. The review 
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established that, during the past few years, most EU member states have begun 
emphasizing the crucial role of learning that occurs outside of and in 
addition to formal education and training. This emphasis has led to an 
increasing number of political and practical initiatives that have in turn 
gradually shifted the issue from the stage of pure experimentation to that of 
early implementation. (The bibliography lists 62 references. A glossary is 
appended.) (MN) 
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preface 



DURING THE PAST FEW YEARS the issue of identification, assessment and 
recognition of non-formal learning has been brought to the forefront of 
European debates on education, training and learning. At different levels, 
spanning from single enterprises to national ministries, initiatives have 
been taken to assess competences acquired outside formal education and 
training. In some cases, the main purpose has been to ‘open up’ existing 
systems of education and training for alternative learning pathways while 
trying to avoid unnecessary repeat learning sequences. In others, the 
purpose has been to support lifelong learning, to make it possible for 
individuals to capitalise on various learning forms and pathways, for 
example, through a combination of learning in school and at work. In yet 
other cases, the identification and assessment of non-formally acquired 
competences has been used as a tool for improving the quality of learning 
in enterprises and organisations. 

This report presents an overview of European initiatives in this area. 
The presentation is based on 14 national reports commissioned by 
Cedefop during the period 1997-99- With the exception of Luxembourg, 
these reports cover all the Member States of the EU. In addition, a 
substantial amount of information has been gathered from other sources, 
including the EU Commission and various programmes supported by the 
EU. A draft version of the report was presented to an international 
conference on the topic arranged by Cedefop in cooperation with the 
Norwegian Ministry of Education and Research in May 2000. The feedback 
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received during this event, where representatives of 18 European countries 
were present, has been integrated into the current final version of the 
report. 

The report should be read as one of several possible interpretations of 
developments in this area. The aim has been to draw attention to main 
trends as well as point to the most crucial challenges facing actors in this 
highly innovatory and still unsettled field. 

We hope the report will contribute constructively to future European 
work on questions regarding methodologies and systems for identification, 
assessment and recognition of non-formal learning. 




Stavros Stavrou 

Deputy Director, Ceclefop 
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executive summary 



THIS CONTRIBUTION TREATS the question of how to make learning, 
which takes place outside formal education and training institutions, more 
visible. While learning in the formal education and training system is a 
distinct feature of modern societies, non-formal learning is far more 
difficult to detect and appreciate m . This invisibility is increasingly 
perceived as a problem affecting competence development at all levels 
from the individual to the society as a whole. 

During the past few years, most Member States of the EU have 
emphasised the crucial role of learning that takes place outside of and in 
addition to, formal education and training. This emphasis has led to an 
increasing number of political and practical initiatives, gradually shifting 
the issue from the stage of pure experimentation to that of early 
implementation. 

Identification, assessment and recognition of non-formal learning has 
to be based on simple and inexpensive methodologies and a clear notion 
of how institutional and political responsibilities are to be shared. But first 
and foremost, these methodologies have to be able to deliver what they 
promise, with the quality of ‘measurement’ being a crucial aspect. This 
report makes an effort to clarify, through an initial theoretical discussion, 
the requirements for reaching successful practical solutions. 



(1) The term non-formal learning encompasses informal learning which can be described as unplanned 
learning in work situations and elsewhere, but also includes planned and explicit approaches to 
learning introduced in work organisations and elsewhere, not recognised within the formal 
education and training system. 
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The character of learning 

When approaching the questions of how to identify and assess non-formal 
learning it is crucial to keep in mind that learning is contextual in its 
character. When taking place in social and material settings, knowledge 
and competences are very much the result of participation in ‘communities 
of practice’. Learning cannot be reduced to passive reception of ‘pieces’ of 
knowledge. This perspective implies a focus not only on the relational side 
(the role of the individual within a social group) but also on the negotiable, 
concerned and engaging nature of learning (the communicative character 
of learning). The individual learner acquires the skill to perform by actually 
engaging in an ongoing process of learning. Learning is thus not only 
reproduction, but also reformulation and renewal of knowledge and 
competences. 

The results of learning processes, what we call competences, are partly 
tacit (Polanyi 1967) in their character. This means that it is difficult to 
verbalise and delimit the single steps or rules intrinsic to a certain 
competence. In some cases, people are not even aware of being in 
possession of a competence. This is highly relevant to the task of assessing 
non-formal learning and has to be reflected by the methodologies. Much of 
the know-how which we possess was acquired through practice and 
painful experience. An experienced carpenter knows how to use a tool in 
ways that escapes verbalisation. Normally, we take this know-how so 
much for granted that we do not appreciate the extent to which it pervades 
our activities. 

I— I Methodological requirements 

The important issue is whether it is possible to develop methodologies able 
to capture the (contextually specific and partly tacit) competences in 
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question? While specialised methodologies for assessment of non-formal 
learning still have a long way to go, testing and assessment within formal 
education and training can refer back to a long history of practice, research 
and theory. The ongoing expansion of assessment into work and leisure 
time is inevitably linked to this tradition. It may be assumed that new 
approaches rely heavily on the methodologies developed within the more 
staictured learning areas presented by formal schooling. At least it may be 
assumed that some of the same challenges and problems are shared 
between the two learning domains. 

Assessment in formal education and training can be said to serve two 
main purposes. The formative purpose is to aid the learning process. No 
system can function properly without frequent information on the actual 
working of the process. This is important in classrooms as well as in 
enterprises: the more variable and unpredictable the context, the more 
important the feedback. Ideally, assessment should provide short-term 
feedback so that learning deficits can be identified and tackled 
immediately. The summative purpose is to provide proof of an 
accomplished learning sequence. Although these proofs may take many 
forms (certificates, diplomas, reviews, etc.) the purpose is to facilitate 
transfer between different levels and contexts (from one class to another, 
from one school to another, from school to work). This role can also be 
formulated as one of selection and a way of guarding the entrance to 
levels, functions and profession. 

The confidence attributed to a specific assessment approach is 
generally linked to the criteria of reliability and validity. The reliability of 
an assessment depends on whether results can be reproduced in a new test 
occasion and by new assessors conducting the test. Validity can, in many 
respects, be looked upon as a more complex concept and concern than 
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reliability. A starting point might be to consider whether an assessment 
measures what it was originally intended to measure by those preparing it. 
Authenticity is a primary concern; high reliability is of little value if the 
result of the assessment presents a distorted picture of the domain and 
candidate in question. 

Reliability and validity are meaningless concepts, however, if not linked 
to reference points, criteria for judgement and/or standards of achieve- 
ment, etc. We can identify two main principles used when setting these 
reference points and/or criteria. In formal education and training, norm 
referencing (according to the setting of a group) is commonly used. The 
second way of establishing a reference point is to relate a given perform- 
ance to a given criterion. Criterion-referenced testing implies identifying a 
domain of knowledge and skills, then trying to develop general criteria on 
the basis of the performance observed within this specific domain. 

The lessons from testing in the formal system can be used to raise a 
number of questions and topics relevant to the domain of non-formal 
learning: 

(a) which functions, formative or summative, are to be fulfilled by the 
new methodologies (and institutional systems) for identification, 
assessment and recognition of non-formal learning? 

(b) the diversity of learning processes and learning contexts raises the 
question of whether the same kind of reliability can be achieved in 
this area as in formal education and training. 

(c) the contextual and (partly) tacit character of learning complicates 
the quest for validity and the question is whether methodologies 
are properly designed and constructed in order to deal with this 
issue. 

(d) the matter of reference points (‘standards’) is a key issue which 
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needs to be addressed. The question is whether domain bound- 
aries (including ‘size’ and content of competences) are defined in a 
proper way? 

It is an open question whether assessments of non-formal learning 
implies the introduction of new tools and instruments or whether we speak 
of old approaches to new challenges. There is reason to believe that to a 
certain degree we at least face a transfer of traditional testing and 
assessment methodologies into this new domain. 

□ Institutional and political requirements 

The future role of systems for the assessment and recognition of non- 
formal learning cannot be limited to a question of methodological quality. 
While being important, reliable and valid methodologies are not sufficient 
to make individuals, enterprises and/or educational institutions trust and 
accept assessments. This is particularly the case if assessments are given a 
summative role, providing a competence proof to individuals competing 
for positions in the labour market and in educational institutions. A number 
of political and institutional preconditions have to be met to attribute some 
actual value to the assessments in question. This can be done partly 
through political decisions securing the legal basis for initiatives but should 
be supplemented by a process where questions of ‘ownership’ and 
‘control’ as well as ‘usefulness’ must be clarified. In this way, assessments 
of non-formal learning would be judged according to technical and 
instrumental criteria (reliability and validity), as well as normative criteria 
(legality and legitimacy). Furthermore, acceptance of assessments of non- 
formal learning is not only a matter of their legal status but also of their 
legitimacy. 
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National and European experiences 

The European situation is presented through examples of five country 
clusters and activities at EU level. Although countries within each cluster 
may differ somewhat in their methodological and institutional approaches 
and choices, geographical nearness as well as institutional closeness seem 
to motivate mutual learning and to a certain degree common solutions. 

□ Germany and Austria; the dual system approach 

The German and Austrian approaches to the question of identification, 
assessment and recognition of non-formal learning are very similar. It is 
interesting to note that the two countries where work-based learning has 
been most systematically integrated into education and training (through 
the dual system) have so far been very reluctant to embrace this new trend. 
On the one hand, this reflects success; the dual system is generally viewed 
as successful both in terms of pedagogy (the combination of formal and 
experiential learning) and capacity (high proportions of the age groups 
covered). On the other hand, and reflecting the strong emphasis on initial 
training, the existing system seems only partly able to extend its functions 
to continuing vocational training and to the more diverse-training 
requirements of adults. But, in spite of this, we can observe, a -substantial 
amount of project-based experimentation and the attention towards these 
questions is increasing. The discussion on recognition of non-formal 
learning in Germany and Austria is closely linked to the discussion on 
modularisation of education and training. 
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□ Greece, Italy, Spain and Portugal; 
the Mediterranean approach 

The general attitude to the introduction of methodologies and systems for 
non-formal learning in Greece, Italy, Spain and Portugal is positive. Both in 
the public and private realms, the usefulness of such practices is clearly 
expressed. The huge reservoir of non-formal learning which creates the 
basis for important parts of the economies in these countries needs to be 
made visible. It is not only a question of making it easier to utilise existing 
competences, but also a question of how to improve the quality of these. 
Methodologies for the assessment and recognition of non-formal learning 
can be viewed as tools for quality improvement, encompassing not only 
single workers and enterprises but whole sections of the economy. These 
countries also illustrate that the step from intention to implementation is a 
long one. Legal and political moves have been made through educational 
reforms of varying scope but the actual introduction of assessment and 
recognition practices has not progressed very far. The coming years will 
show whether the positive intentions almost unanimously expressed in the 
four countries will be translated into practices which actually affect and 
serve individuals and enterprises. 

□ Finland, Norway, Sweden and Denmark; 
the Nordic approach 

It is not possible to speak of a ‘Nordic model’ at least not in any strict sense. 
Finland, Norway, Denmark and Sweden have chosen different approaches 
and are working according to somewhat different schedules. These 
differences do not change the fact that all four countries have taken 
practical steps through legislation and institutional initiatives, towards 
strengthening the link between formal education and training and learning 



making learning visible 



taking place outside schools. Despite the fact that some elements of this 
strategy have existed for some time, the most important initiatives have 
taken place in recent years, mostly since 1994-95. The mutual learning 
between these countries is strong and has become even stronger over the 
past two to three years. The influence of Finnish and Norwegian 
approaches on recent Swedish documents illustrates this effect. Finland 
and Norway are clearly opening up for the institutional integration of non- 
formal learning as part of a general lifelong learning strategy. The plans 
presented in Sweden and Denmark indicate that these two countries are 
moving in the same direction and that the issue of non-formal learning will 
become more focused in the coming years. 

□ UK, Ireland and the Netherlands; the NVQ approach 

In the UK, Ireland and the Netherlands we can observe strong acceptance 
of an output-oriented, performance-based model of education and 
training. General acceptance of learning outside formal education and 
training institutions as a valid and important pathway to competences is a 
basic feature in these countries. What is questioned, however, is how such 
a system should be realised. The UK and Dutch experiences illustrate some 
of the institutional, methodological and practical problems associated with 
establishing a system able to integrate non-formal learning within its 
framework. The challenge of developing an acceptable qualification 
standard seems to represent the first and perhaps most serious obstacle. As 
long as assessments are supposed to be criterion-referenced, the quality of 
the standard is crucial. The UK experiences identify some of these 
difficulties balancing between too general and too specific descriptions 
and definitions of competences. The second important challenge 
illustrated in the UK and Dutch cases, but not reflected in our material on 
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the Irish experience, is related to the classical assessment challenges of 
reliability and validity. In our material the problems have been clearly 
demonstrated but the answers, if they exist, are not so clearly defined. All 
three countries base their vocational education and training on 
modularised systems, a factor which seems to support the rapid and large- 
scale introduction of methodologies and institutions in the field. 

□ France and Belgium; ‘opening up’ diplomas and certificates 

In several respects, France can be characterised as one of the most 
advanced European countries in the area of identification, assessment and 
recognition of non-formal learning. Belgium has been less active, but a 
number of initiatives have been taken during recent years, partly 
influenced by the French experiences. The first French initiatives were 
taken as early as 1985 when the system of the bilan de competence was 
introduced. The aim of the bilan is to support the employer/employee in 
identifying and assessing professional competences; both to support career 
development and enterprise-internal utilisation of competences. 

The second important French initiative was the 'opening up’ of the national 
vocational education and training system for competences acquired 
outside formal institutions. Since 1992, vocational certificates ( Certificate 
cl’aptitude professionelle ) can be achieved (to various degrees) on the basis 
of assessments of non-formal and prior learning. A third important initiative 
was taken by the French chambers of commerce and industry where the 
aim was to set up procedures and standards for assessment independent of 
the formal education and training system. Using the European norm 
EN45013 on procedures for certifying personnel as a point of departure, 
important experiences have been gained. Parallel activities based on EN 
45013 are going on in Belgium. 



making learning visible 



□ EU approaches 

Initiatives at European level have clearly been important in pushing the 
issue forward in the minds of the public and politicians. The white paper 
on teaching and learning (1995) helped to define the issue in a clear way 
and thus supported the processes at national and sector levels. The 
resulting programmes (mainly Leonardo da Vinci and Adapt) have initiated 
and financed unparalleled experimental activity. While not interfering 
directly in efforts to develop national systems, the EU has clearly increased 
interest in the issue and also contributed in a practical sense by supporting 
methodological and institutional experimentation. This does not mean that 
the particular strategy of the white paper, focusing on European standards 
and a European personal skills card (PSC), has been implemented. One 
important reason for this is the mixing of objectives in the original 
conception of the task. On the one hand, the PSC was presented as a 
summative approach; introducing new and more flexible proof of 
qualifications and competences. On the other hand, the need for new 
assessment methodologies was promoted on the basis of the need to 
identify and utilise a broader basis of competences; what we may term a 
formative objective basically addressing the support of learning processes. 

Looking into the Leonardo da Vinci experiment, the first objective has 
only been elaborated and followed up to a limited degree. Where a 
summative element can be detected, it is normally with a clear reference to 
existing national qualification systems or linked to a limited sector or 
profession. The formative aspect, however, has turned out to be a main 
concern. Not in the form of extensive supranational systems, but in the 
form of practical tools for single employers and/or employees. Opening up 
for initiatives from a wide variety of actors, questions and methodologies 
have been initiated at a low’ institutional level where formative issues and 
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concerns have dominated. Or, to put it another way, the activity of the 
projects illustrates the priorities of enterprises and sectors, not the priorities 
of national ministries. 



Why focus on non-formal learning? 

What has triggered this wave of activity affecting most European countries 
almost simultaneously? Answering this requires focusing on political and 
institutional objectives, developments and challenges. Below, we will 
emphasise three aspects. 

□ Reengineering education and training 

To establish a system for learning throughout life requires a stronger focus 
on the link between different forms of learning in different learning 
domains at different stages of life. While the formal system is still very 
much focused on initial education and training, a lifelong learning system 
has to face the challenge of linking a variety of fonnal and non-formal 
learning areas together. This is necessary to meet the individual’s need for 
continuous and varied renewal of knowledge and the enterprise’s need for 
a broad array of knowledge and competences - a sort of knowledge 
reservoir to face the unexpected. The question of identification, assessment 
and recognition of competences is also crucial. Competences have to be 
made visible if they are to be fully integrated into such a broader strategy 
for knowledge reproduction and renewal. 
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□ Key qualifications 

Although normally treated as two separate issues, the question of how to 
define, identify and develop key qualifications and the challenge of how to 
assess non-formal learning are closely related. We will argue that these two 
debates reflect different aspects of the same issue. In both cases we can 
observe increasing attention towards learning and knowledge 
requirements in a society characterised by unprecedented organisational 
and technological change. Methodologies and systems for identification, 
assessment and recognition of non-formal learning can be looked upon as 
practical tools for making key qualifications visible and stronger. The terms 
informal and non-formal learning are, however, not very helpful in this 
respect. Non-formal learning is a ‘negative’ concept in the sense that it is a 
negation of something else. It gives little positive indication of content, 
profile or quality. The concept is important, however, by drawing attention 
to the rich variety of learning areas and forms available outside formal 
education and training. A closer link to the key qualification issue might 
thus be useful and give the exercise more direction. The linking of formal 
and non-formal learning domains can be viewed as a way of realising and 
materialising the objectives expressed through key qualifications. 

□ Solutions seeking problems; 

a supply driven development? 

Only in a few cases can the development of measurement and assessment 
methodologies be described as driven by demand or by a push from the 
bottom up. If we study the last half of the 1990s when this trend gained 
momentum and strength the existence of programmes like Adapt and 
Leonardo da Vinci at European and sector levels have contributed to the 
setting and changing of ‘the assessment agenda’. The availability of ‘fresh 
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money,’ linked to a limited set of specific priorities, inspired a high number 
of institutions to involve themselves in the development of instruments and 
tools. Although the results of these projects may be of varying quality, the 
long-term impact on the agenda of the organisations and institutions 
involved should not be underestimated. The coming period will show 
whether this supply driven movement will find users, for example at sector 
and enterprise levels, appreciating the effort put in. 



How should we identify, assess 
and recognise non-formal learning? 

Answering the question of why interest in non-formal learning has 
increased does not provide an answer to the question of how to support 
and strengthen the positive elements of these developments. Following the 
theoretical clarifications made in the first part of the report, the challenges 
ahead can be defined as both a methodological (how to measure) and a 
political/institutional one (how to secure acceptance and legitimacy). 

□ Methodological requirements 

Which functions are to be fulfilled by new methodologies (and institutional 
systems) for identification, assessment and recognition of non-formal 
learning? Do we speak of a formative role where the instruments and 
tools are used to guide the learning processes of individuals and 
enterprises or do we speak of a more limited summative role where non- 
formal learning is tested for possible inclusion into the setting of formal 
education and training? The purpose of the assessments, in the non-formal 
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as well as in the formal domain, is decisive for the methodological 
choices to be made and for the ultimate success of the exercise. Successful 
development of methodologies and systems implies that these functions 
are clearly understood and combined and/or separated in a constructive 
and realistic way. • 

The diversity of learning processes and contexts makes it difficult to 
achieve the same kind of reliability as in standardised (for example, 
multiple choice) tests. The question is how (and which specific kind of) 
reliability should be sought in this new domain. Reliability should be 
sought by seeking optimal transparency of the assessment process 
( standards , procedures, etc.). Reliability could also be supported through 
implementation of systematic and transparent quality assurance practices 
at all levels and in all functions. 

The highly contextual and (partly) tacit character of non-formal 
learning complicates the quest for validity. There is an acute danger of 
measuring something other than what is intended- The main thing is to 
avoid a distorted picture of the candidate and the domain and to strive for 
authenticity. Methodologies have to reflect the complexity of the task at 
hand; methodologies must be able to capture what is individually and 
contextually specific. 

The question of reference points (‘standards’) is a major issue for 
assessment of formal as well as non-formal learning. While norm- 
referencing (using the performance of a group/population) has not been 
seriously discussed in the ’context of assessing non-formal learning (due to 
the diversity of competences involved), . the issue of criterion or domain- 
referencing lies at the heart of the matter. The definition of boundaries of 
competence-domains (their size and content) and the ways in which 
competences'can be expressed within this domain is of critical importance. 
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The wider the area, the greater the challenge in designing authentic 
assessment approaches. This reverts, in many ways, to the question of 
functions to be fulfilled; do we want to improve learning processes or do 
we want to produce proof (papers of value)? Both purposes are highly 
legitimate and useful. The setting up of reference points will, however, 
differ considerably according to the purposes selected. 

D Political and institutional requirements 

As soon as the first methodological requirement has been met, by 
answering the questions of methodological purpose and function (see 
above), institutional and political implementation could be supported 
along two main strategies; one focusing on ‘institutional design’ and the 
other on ‘mutual learning’. 

Institutional design : some basic criteria must be fulfilled if proof of 
non-formal learning is to be accepted along with proof of formal education 
and training. First of all, participants must be heard when setting up and 
operating systems of this kind. Since systems for recognition of non-formal 
learning will have a direct effect upon the setting of wages as well as on 
the distribution of jobs and positions in the labour market, this matter 
clearly incorporates the balancing of interests. Although not emphasised 
very much until now, the question of who to involve and who to listen to 
will be of decisive importance in the coming period. Secondly, relevant 
information must be fed into the process. On the question of represent- 
ation, the definition and articulation of standards and reference points (in 
particular) require sufficient and balanced information. Thirdly, the 
transparency of the structures and procedures are very important. It is 
possible to establish structures where the division of roles (setting of 
standards, assessment, appeal, quality control) is clearly defined and 
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presented. Transparency of procedures is ‘a must’ if acceptance and 
legitimacy are to be achieved. The attention of both researchers and policy- 
makers must be drawn to all these issues in the near future. 

Mutual learning should be sought and supported between projects, 
institutions and countries. A substantial amount of learning is already 
taking place at various levels. As concluded in other parts of this report, 
and especially in relation to activity at European level, the potential for 
mutual learning is much greater than the actual and factual achievements 
thus far. Establishing such learning mechanisms must reflect the various 
purposes and functions to be fulfilled. Finally, it is very necessary to 
increase coordination and to support activities (at European and national 
levels) in order to capitalise on the experiences gained through numerous 
existing projects, programmes and institutional reforms. 
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